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The paper outlines the main defects and flaws of the current system and 
practice of appraisal in science. First of all, ‘closedness’ and non- 
transparency of the procedures of evaluation expand the arbitrariness of 
experts and lower their responsibility. Further, standardization and 
formalization lead to depersonalization and, there through, to emasculation 
of expert judgments. And finally, monopolism in holding the “control 
sticks” and “communalism” of professional life bring almost to naught the 
personal dimension of science, subjecting it to the organizational hierarchy. 
In the issue, experts turn out to be fundamentally dependent and driven 
into the framework of bureaucratic order. Under such conditions, it is hard 
to reckon on unprejudiced evaluations. As regards the systematic support 
of radically new ideas, their effective boosting and growth, that proves to 
be an idle groundless fancy. 


1 In the November 2016 edition of this journal, the author continues the theme of this first part (1 of 2) by examining What to Do to Get out 
of the Impasse of Neophobia? 
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Introduction 

The resistance to new ideas is unavoidable. It is the charge for their going out of the intellectual 
frontier of epoch and, there through, of the horizon of the vast majority of experts who hardly 
“digest” what does not fit into the canons they have learnt. To imbibe an unwanted idea it is not 
enough to have some general knowledge scientists possess. Little good comes of specialization, 
too, unless it keeps pace with the new views. The profundity of understanding only narrows the 
“cone” of vision and ousts from the sight all not going into the Procmstean bed of established 
conceptual patterns. And therefore, in order to cope with something radically new it is necessary 
not merely to go deep into its marrow but also to wander from the beaten roads, cast away the 
standard forms of interpreting phenomena, what implies tearing away from the bulk of colleagues. 
But every lead is fraught with alienation and miscomprehension, and the more they are the stronger 
the resistance to the idea advanced. 

Nevertheless, despite the harm coming from the bigotry and obscurantism and even from mere 
following the tradition, it would be unwise to eradicate the opposition to the new, leaving before 
the latter no barriers and filters. And the point is not only that many a novelty is far from being 
reasonable and viable. Human mind has such an arrangement that cannot turn off the aversion to 
the unfamiliar and unexplored without undermining its own critical function. When anything fresh 
and unusual suits, everybody may utter whatever comes into head. The more wild the idea and the 
farther from the knowledge on hand and the common sense the higher is the assessment it should 
receive. But such an approach clearly does not favor the progress of the mankind and, moreover, 
threatens it with collapse. 

However under swift changes in society, the conversion of science into an immediate productive 
force, and the stiffened competition between research groups, corporations, and states for making 
technical leverages and social technologies of manageable renewal, innovations appear to be not 
only the products of creative activity but also the objects of planning. They get purposively 
prepared and introduced into practice. Under these conditions expressing the peculiarity of the so- 
called “big science”, new ideas must be not merely encouraged and countenanced; they should be 
brought up and fostered, for it is just them that underlie, ultimately, the innovative breakthroughs. 

The most obvious and, at the same time, the most protracted way to slackening the resistance to 
innovations is the change of the culture of handling the new. It needs to inculcate in people at the 
psychological level, to build into their mentality more tolerance toward the non-habitual and 
unexpected, and — what is even more important - higher criticism toward the reputed knowledge. 
It does not mean to put the questionable on a level with the time-tested. Merely what is non- 
proved as yet can turn out to be quite efficient and useful just as the seemingly well-established 
delusive and fruitless. 

Much easier and quicker could be secured the support for new ideas should the ambience reigning 
in organizations and communities fundamentally change. Through upholding the free flow of 
opinion, respect for alternative views, and above all, ousting to the fringes unsubstantiated 
statements and authoritarian “prophesying” which substitute argumentation and corroboration for 
speechcraft and psychological pressure, the path of new ideas from their birth to consolidation in 
the minds of scientists would get essentially smoother. But whatever favorable for perceiving and 
disseminating them the ambience may be, it will get one nowhere while the organizational 
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framework within which the “sentences” upon new ideas are passed remains the same. The very 
practice of expert evaluation in “big science”, of giving appraisal of research findings and 
contributions to science appears to be a factor restraining any radical renewal. And while it keeps 
within its current form, one scarcely may reckon on manageable and systematic innovative growth. 

What is the mechanism of inhibiting the new? Which parts it comprises and how acts? What 
changes it is worth making in the guidelines, ways, and procedures of appraisal in order to give 
original ideas more essential support or, at least, to lower the opposition to them? Here are the 
main questions this paper tries to answer. 

Blind Review 

Among the factors exerting a smothering influence on new ideas, burst upon the eye, first and 
foremost, the closedness and non-transparency of the procedures of assessment and decision- 
making. Many consider them a merit, though. Those responsible for evaluation are implied to be 
allegedly guarded from external pressure and direct manipulation by interested forces. And thus, 
as it is supposed, the independence and impartiality of decisions made by experts get secured. 

So, dissertations are defended at ballot box. It remains unknown not only how one has voted but 
what are the reasons of that decision. Passing judgment on the degree-seeker as well as on the 
ideas he has advanced, the judge does not have to justify his own position or even to possess it. 
No surprise, in many a case, the voting does not evidence the actual attitude toward the subject 
being assessed. Moreover, sometimes the assessment is made without any acquainting with it. 

Something of the kind occurs in the Commissions and Committees deciding on grants and 
scholarships. The real virtues of the project or of the creative potential of its author are in no way 
taken into account primarily. The accountability of their members to the general public is not 
provided for at all. As to the position each of them stands for, even if it is somehow corroborated, 
that gets known only to a narrow circle. Consequently, nothing makes him be concerned with the 
justness or profoundness of evaluation. 

But the apotheosis of incompetence and arbitrariness is seen in the practice of reviewing papers 
by scientific journals, which has almost ceased to conform to its initial purpose to such a degree 
that one scarcely grasps what it exists for. In most publications, they boast of employing the so- 
called double-blind review when the author and the reviewer have allegedly no notion of each 
other. Its purport is the impartiality of reviews, what, in turn, seemingly should enable more free 
passing of new ideas. 

Meanwhile, this is an abysmal delusion. Depersonalizing reviews turns them into absolutely 
irresponsible and inane texts. The reviewers often than not allow themselves to make arbitrary 
remarks and comments bolstered with nothing, and abandon going deep into the essence of the 
issues being discussed. Moreover, their work changes into a mere rite having nothing in common 
with either analysis or sound criticism. 

Frequently, journals do not take the trouble to give elaborate assessments. Furthermore, the papers 
received are far from being always sent out for reviewing. So, the article on the cycle of cell division 
later published in “Science”, where for the first time the ideas which eventually fetched in 2001 
the Nobel Prize for Hartwell had been considered (Hartwell, Culotti, et al, 1974), “Nature” 
discarded without reviewing and explanation. As Pringle, one of the co-authors, recollects, that 
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left “a bad taste” for long, and he had “never again submitted a paper there by choice” (Pringle, 
2013; 3283). 

Quite often, instead of sticking to the point, editors try to get away with figures of speech that 
mean nothing as such but serve as a kind of soft refusal. For instance, the Higgs’s brief note 
forwarded to “Physics Letters”, where he was explaining the sources of the sub-atomic particles’ 
mass (the idea awarded with the Nobel Prize of 2013) had been turned down for the reason that 
it could not be published in near future (Griggs, 2008; 17). The editors had not thought fit to study 
in depth the material and resorted to so ornate-diplomatic wording merely to brush away the 
author. 

Taschner regards as illusion the belief that those making decisions on publication thoroughly 
familiarize themselves with the text. At best, they address a request (by phone or via e-mail) to 
their good friends from among those who are in the know to get an elucidation whether the author 
is worth supporting. The state science has turned out to be in seems to Taschner pitiful and 
cormpting (Taschner, 2007). And to tell the tmth, it is hard to disagree with him. 

To make sure that even apparently reputable editions allow themselves perfunctory and 
irresponsible reviewing it is enough to turn to conspicuous facts. In mid-90s of the past century, 
much ado have been produced by the story of the physicist Sokal who composed a parody of 
culturological paper called “Transgressing the Boundaries: Toward a Transformative 
Hermeneutics of Quantum Gravity” (Sokal, 1996). He submitted it to a well-known journal 
(“Social text”) under a respected institution (Duke University). And to the author’s amazement, 
that paper had been soon issued without any suspicion on the part of the editors. 

Meanwhile, the material was rather peculiar and looked quite funny. A special savor was zested to 
it by that the most ridiculous passages belonged not to the author; they had been borrowed from 
post-modernists. To put it more precisely, he had built his paper on the almost directly quoted 
utterances of eminent French and American intellectuals about the philosophy of physics and 
mathematics. His own “contribution was to invent a nonsensical argument linking these quotations 
together and praising them.” It implied surely advocating what looked like “an incoherent 
mishmash of trendy ideas”, as well (Sokal, 2010; 153). But that only added spice to the parody. 

Why the editors missed the “arrowhead”? Moreover, they had even not grasped that the author 
scoffed at them. In what way the editors were evaluating the text? And had they acquainted 
themselves with it at all? 

Undoubtedly, nobody went into the substance of the paper. But evidently a kind of marginal 
acquaintance with it took place, yet. The editors had enough to see a text that gratified their 
ideological prejudices and attacked the opponents. As Robbins, one of them, somewhat later 
admitted, they saw “a progressive scientist, a physicist who was willing to be publicly critical of 
scientific orthodoxies” (Robbins, 1996; 28). As regards the scientific “filling” of the text the very 
fact that the author was a certified physicist with a degree from a respected university seemed to 
be the pledge of its high level. With hands down, they had gotten a qualified ally - what more 
could be longed-for? But where is the objectivity, impartiality, or precise and balanced evaluation? 

May be, this is a solitary instance, and the real practice of reviewing in journals keeps, on the whole, 
to other procedures and standards? It is by no means so. As to new ideas the contrary cases appear 
to be, rather, exceptions. But even the latter embrace mainly “temperate” novelties. 
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Prior to Sokal, two American researchers, Peters and Ceci, attempting to make out the practice of 
reviewing in psychological journals, devised a simple but instructive experiment. They chose 12 
papers by those who represented the departments of psychology at leading universities and, on 
slighdy changing the texts as well as the names and affiliations of the authors, dispatched the 
manuscripts to the same journals where they had been recently (within the last 18-30 month) 
reviewed and issued. It should seem a storm of indignation directed at “plagiarists” would blow, 
or, at least, the new reviews would be like those received previously. But there was nothing of the 
kind. 

Only in 3 instances the “forgery” had been detected. In 8 out of the rest 9, that is, in 89 percent, 
the papers had been discarded on their demerits (Peters, Ceci, 1982). What changed in them as 
compared with their initial submission? Only one thing — the authors looked much less imposing. 
Instead of big names from prominent universities, “indistinct” ones from godforsaken (more 
exacdy, concocted) “backwater districts” appeared on the editorial tables. No wonder they had 
been cut down to size. 

Is it a scandal? Of course. But an ordinary, common one. And the point is not that reviewers fairly 
often afford swagger and conceit in respect of the colleagues from the “scientific periphery”. All 
is much prosier. They ask themselves: do these “provincial geniuses” or, to put simpler, “parvenus” 
have a sufficient educational background, technical opportunities for elaboration of the problem, 
and intellectual milieu to discuss them in order to come to well-founded conclusions? Since the 
answer is in most cases negative, the attitude to them turns out to be primordially biased and does 
not imply any substantial analysis of the material. In other words, the results of evaluation are 
ready even before the procedure itself begins. 

In the light of this, it does not look astonishing that prestigious awards seldom go beyond the 
bounds of narrow (“elite”) circles, a striking organizational continuity being observed in their 
distribution, reminding of hereditary dynasties. So, by estimation of Zuckerman, about a half of 
all scientists who were conducting their research in USA by 1972 and afterwards conferred with 
Nobel Prizes represented only 5 universities (Berkeley, Chicago, Columbia, Harvard and 
Rockefeller) which comprised 3 percent of the American university staff. Furthermore, over 53 
percent of those (49 out of 92) began their research activity (as students, post-doctorates, or junior 
collaborators) under the guidance of other Nobel Prize-winners. What prevails here - a powerful 
attraction of talents to each other or simply the narrowness of the channels of promotion to getting 
acknowledgement? 

Zuckerman herself was explaining this tendency by that “the masters' own performance provided 
a model to be emulated”; they “evoked excellence from the apprentices working with them” and 
taught “less by precept than by example”. Due to “demanding standards of work”, the masters 
succeeded in sustaining “moral authority” and, therefore, corrected the followers when they 
wandered from the behavior samples (Zuckerman, 1977; 126). Certainly, these are real factors and 
they actually favor the improvement of the quality of research. However there are a lot of such 
exemplary masters - and frequently not less talented - also beyond those select universities as well 
as outside the composition of those who had ever been nominated for Nobel Prize. Nonetheless, 
they rarely succeed in fostering their “own” laureates. 

The case gets clearer when one looks at it from another side. Who is given the largest funding? 
Whose research needs are bolstered with resources in a great extent and most readily? It comes up 
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that in 1962, 38 percent of all American federal support went just to 10 institutions, and if to take 
25 the most funded they attracted even 60 percent (Barber, 1966; 63). Naturally, their scientists 
had much more chances to obtain quality findings. Not to mention that, owing to their better 
“visibility” and authority of the institutions they were associated with, the researchers from those 
select universities could hit the pages of respected journals and raise the ideas advanced higher in 
the public’s eyes. As regards staying in the “narrow channel” of promotion to recognition it only 
crowns the work. 

Sure, the name and affiliation are far from being an exhaustive or the most commonly encountered 
reason for bias. No less essential role is played by conceptual discrepancies, belonging to diverse 
schools of thought, commitment to alternative methods of research, and so on. When it comes to 
new ideas the number of like acting factors increases, and they get miraculously interknit, making 
such a synergy against which rare reviewer can stand. And closedness and non-transparency of the 
procedure of reviewing produce that very breeding ground for prejudice on which the smothering 
of radically new ideas becomes not merely possible or even quite probable but virtually 
unavoidable. All potential menaces expert evaluation is fraught with, stretch out and transmute 
into a straight-jacket for the progress of knowledge. 

One could yet understand if authors (candidates), the weak side of the interrelation, were protected 
against reviewers, that is, were unknown to them. Authors mostly cannot get to know who the 
reviewers are, while to the latter, it is not worth a dime to have ascertained whose contribution is 
being assessed. But what is the advantage gained from concealment of the reviewer? That put him 
in a semi-transparent (with one-way visibility) box where he may be up to all sorts of nonsense 
without risk of getting it on the nose from authors or the general public, what provides an 
inexhaustible source of arbitrary and irresponsible decisions. But that is only one side of the 
problem. 

The other side has much deeper and less obvious roots. Reviewers hardly would obtain an 
opportunity to play pranks at the expense of various institutions (journals, prize committees, 
appointing commissions and the like) if the latter had no interest in such a state of affairs. 
Manipulating the reviewers and orchestrating their behavior (through committing to work, 
instructing, and so forth), those behind achieve own aims without catching the strokes, because 
have no direst deals with authors (candidates), though just on them depends, ultimately, what 
decisions are made. Reviewers themselves as often as not join that game even if they are aware of 
their being a puppet in somebody else’s hands, since obtain a certain status, remaining, at that, 
beyond the reach of those who stands in the breach. Thus reviewers turn into that very 
“untethered” canon which begins to shoot at own forces and destroy what it is called upon to 
defend. 
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Fig. 1. The pattern of interactions between author (candidate), reviewer, and those behind the 
process of evaluation. 


The Problem of Quality 

What favors so mass disregard of what many name “the intellectual duty” of experts? Is it not 
possible to make them treat the material under review more profoundly and unbiasedly reveal both 
their flaws and merits? 

A quarter of a century ago, Horrobin offered to return to making sense of what the expert 
evaluation is given for. In his opinion, it can be successfully employed only where the parties 
involved (editors and reviewers) have a distinct notion of their purpose. It seems to many that the 
issue in question is the quality control. But, according to Horrobin, it is much broader and akin to 
the task of physician - sometimes to cure, frequently to release, and always to soothe. Hence, the 
expert evaluation is called upon to facilitate the introduction of improved methods, to ameliorate 
the current state, and to raise hopes for a better future. That embraces both quality control and 
the support of innovations. And where the balance between these two functions gets lost the 
expert evaluation does not comply with its mission (Horrobin, 1990). 

But how the quality comes to light with the aid of reviewing? What the charge of expert having to 
assess, for instance, a paper submitted to a journal consists in? Should he revise the conclusions 
the author has drawn? Scarcely so, for there is neither time nor possibility for it. May be, the long 
chains of mathematical calculations are to be repeated? Quite dubious. Or to reproduce the 
experiments described in the paper? It is a complete utopia at all. 

Furthermore, in practice, the experts miss even the gross errors and seemingly obvious flaws. In a 
research, their ability to catch mistakes in the text had been examined. A paper on neurobiology 
containing 8 major “introduced” defects was sent out to 420 reviewers. In average, they were a 




International Journal of Innovation, Creativity and Change, www.ijicc.net 
Volume 2, Issue 3, May 2016 



success in 2 instances. At that, no one of experts could detect more than 4 blunders, while 16 
participants had found none (Rothwell, Martyn, 2000). 

In another case, the reviewers reported, in average, only about 3 out of 9 flaws. At that, a quarter 
of them contrived to find no mistakes. It is not worth mentioning such “details” as the 
discrepancies between the text and the tables adduced. In the main (in more than a half of 
instances), only the gross boners related to the techniques of sampling and randomization had 
been exposed, while the incongmities in the analysis of data and the presentation of results were 
detected almost twice as rarely (less than 30 percent). Although many a reviewers saw the 
conclusions to go farther than the outcomes received, nearly 40 percent of the experts did not 
notice that the authors extrapolated their results to the areas they did not ever considered. Even 
after a special training, an improvement in detecting the errors occurred chiefly in technical matters 
such as the rate of response or the statistical procedures employed. As to the context of research 
and its linkage to policy or practice no progress had been registered (Schroter, Black, et al., 2008; 
510-511). 

What sieve one should pass the material through to get a notion of its quality? 

Curiously enough, the question of the tasks of reviewing rarely is set in a manifest and explicit 
manner. Nevertheless, summing up the widespread approaches, one may conclude that by it, 
“authorities in a given field determine the validity and assess the relative significance of a particular 
contribution of a scholar or scientist within that field” (Osburn, 1989; 279). Much worse stands 
the matter with the criteria the fitness of the text for publication is defined by. They nowhere are 
described in fine details - at least as some universal rules. Even if so “yardsticks” exist in the minds 
of reviewers they appear in a rather fuzzy and blurred semblance. And when reviewers are asked 
to formulate these criteria the answer sounds mostly unconvincing: sure, it is hard to make 
unambiguous demands as well as to draw a neat border-line between the acceptable and the 
inadmissible, but at least, so absurdity as what Sokal has stated in his notorious paper on the non- 
existence of gravitation should be suppressed without remorse (Gernert, 2008; 242). This alone 
would let scientific editions get rid of a bulk of trash. 

All is tme, but the trouble is that a concept having a profound meaning within one conceptual 
system can turn out to be a full nonsense within another. Say, the absolute simultaneity quite 
natural to a person whose mind has imbibed the Newtonian “ideology” will seem to a relativist a 
pitiable survival of the never-to-return epoch. And to the contrary, the relative simultaneity being 
enthusiastically communicated by relativists as an insight looks, for a traditionalist-Newtonian, as 
an outright gibberish. What will do the expert who has no notion of the theory of relativity when 
he receives a paper to be reviewed whose author not only easily operates with this concept but 
also passes it for a precise description of the reality? Will not he decide that the paper embodies 
idle and fruitless fancies not bolstered with facts, and therefore a block should be set on its way? 
The answer is obvious. 

Then, what is the quality of scientific work? 

Apparently, to put aside the clarity and stylistic adjustedness of the text as well as the keeping to 
the norms and standards of making-up befitting the epoch, the matter should be of the value of 
the results obtained by the author. The more it is the higher the quality. And on the contrary, when 
the results possess no particular value, however beautiful the text may be and however closely it 
may meet the formal requirements the level of quality will remain in question. 
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But what is meant by the value of scientific results? 

Broadly speaking, a tacit consensus comes from long ago according to which the quality (value) of 
a scientific paper should be assessed by a three-dimensional scale. Its primary demand is topicality 
(T) what implies that the research responds to some conceptual or practical needs of its time and 
offers a certain solution to the respective problems. As the second component of quality, appears 
the contribution it makes to the knowledge in hand, the accretion which becomes possible due to 
the research accomplished. This is nothing but the newness of the results, their originality (O). 
The higher it is the more the addition science owe to the research under evaluation. Finally, a 
substantial role is played by how far the claims of the author are bolstered with well-working 
theories and established facts. That is usually called the validity of the points advanced. It sets the 
level of confidence in the results and conditions their readiness to being used in science and 
practice. 

Assumed that there is a real opportunity to assess each of the components of quality, and their 
weights (relative importance of each to the evaluator) have been set, one can give a general 
appraisal of the research by means of the quantitative formula: 

Q = a*T + p*0 + /TV, where 

T is topicality, O - originality, V - validity, and a, [3 and X - the weights of the respective 
components. It should seem that this three-dimensional reference frame allows of bringing to light 
and comparing with each other the values of the results of different investigations and serves a 
safe ground for making decisions on which works must be encouraged and which, to the contrary, 
ousted from the foreground of science. 

However the trouble is that these indicators are non-orthogonal, that is, cannot be independent 
variables. Closely knit together, they can both reinforce and undermine each other. Say, when a 
research is topical, that obliquely increases its validity, though validity itself is rather indifferent to 
topicality. On the other hand, originality is poorly compatible with both topicality and validity. The 
more original are the results the less demanded (in the extreme case it is entirely incomprehensible 
how and to what they can be applied). And - what is even more important - the broader the 
newness the worse they get bolstered with the knowledge in hand. 

No wonder. The substantiation of an idea is, in ideal, nothing but stretching to it a continual 
conceptual chain from the concepts being in circulation. The higher the originality of an idea the 
more gaps to be covered and the more saltatory the transitions, and consequently, the less valid 
the idea (Petrosyan, 2015; 173 - 177). Otherwise, high amount of newness automatically decreases 
the level of validity and - to a less degree - the topicality. To the contrary, the striving for 
heightening the topicality and, all the more, validity of the research inevitably emasculates its 
originality. For instance, if the weights of the components are identical the truly innovative works 
will receive no high evaluations, for they are proclaimed to be both in little demand and poorly 
corroborated. And hence, even if they are not suppressed it is, for sure, not worth reckoning on 
their wide recognition. The situation is aggravated by that the colleagues who make judgments are 
not bodiless spirits incarnating a pure and undisturbed yearning for the truth but the living with 
all their merits and demerits, passions and sympathies to the “nearest and dearest”. And when 
personal urges and social inducements become immediate motives for behavior, leaving their 
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stamp on evaluations, they lose the last grains of adequacy and turn out to be beyond both the 
truth and common sense. 



Fig. 2. The alleged (left) and real (right) interrelations between topicality, originality, and validity. 

Thurner and Hanel have built a simple model by which it is shown that the current reviewing 
practice - provided that a group of “rational” experts not wishing any high-quality work other than 
their own to be published take part in it - is simply incapable of securing the quality of scientific 
texts. It comes out that even a small portion of so “wrong” (egoistically-rational) persons being 
not guided by the rules of “correct behavior” (love of truth, impartiality, refusal of nepotism and 
corporate solidarity, etc.) drastically decreases the quality of published or sponsored papers. It 
cannot be essentially higher than at random choice (for example, on drawing lots). Thus, when the 
“quasi-rational”, “right”, and “random” reviewers are presented in equal shares - by 1/3 - the role 
of reviews are virtually leveled. Moreover, the attempts to enhance the policy of quality get fraught 
with further deterioration of the papers accepted for publication (Thurner, Hanel, 201 1; 710 - 711). 

Therefore, the problem is actually not that the reviewers in chase of quality trample down in 
ground the fragile beginnings of new knowledge. The concept of the quality of scientific work 
embraces not only and not so much a conformity with some adopted criteria which, to boot, rather 
sensibly echo with the winnowings of the epoch but also a fairly good portion of originality, that 
is, some proper contribution to science. However that inevitably clashes against the mental tune 
of the expert himself, and when his inner conceptual reality rejects the “currents” coming from 
the material under review, he checks in it everything - the reasons adduced, the correctness of 
reasoning, the completeness of the prehistory the problem has grown out of, the relevance of 
references, and even the personality of the author - except its scientific level. That is why a sciolistic 
stuff has very frequently even more chance to get to journal pages than a genuinely breakthrough 
paper. 

But resisting the innovations, reviewers undermine, first and foremost, the quality of publications, 
their heuristic potential. And till the practice of “blind reviewing” predominates, one may expect 
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no support for conceptual breakthroughs in science. Quite the contrary, new ideas are artificially 
restrained, while the field for discussing and applying them shrinks like shagreen leather. 

Under “blind reviewing”, nobody can blame experts for their low-quality evaluations and biased 
judgments. And when they are reluctant to delve into the sophistications of the author’s thought, 
and at that, have an almost absolute guarantee of no-responsibility for their incompetence and lack 
of professionalism, the first to fall prey are radically new ideas which poorly fit in with the 
automatisms of apprehension and, certainly, get not learnt at careless scanning through. Not to 
mention the cases of outright prejudice when the author or his text are to the reviewer as a red rag 
to a bull, or when an order for a certain review is performed. 

Sure, the organizational closedness is by no means equal to total ousting the novelties. They can 
appear - and sometimes quite eccentric. But only “admissible” innovations will have tried out for 
inclusion in the body of trustworthy knowledge - those with low-grade newness, congener to the 
prevailing views, or coming from people of the “same kin”. 

Standardization of Judgments 

Even a more powerful force egging to resistance to the new is the standardization of judgments. 
It carries on the line of depersonalization begun by closedness and non-transparency and, for good 
and all, erodes the personal responsibility of those who make decisions. That course is based on 
the unification of assessments as well as of the criteria and order of evaluation, what formally 
consolidates the reigning mental patterns which not merely force out to the roadside the fresh and 
unwonted thoughts but virtually impenetrably block the way to discussing them, to say nothing of 
their learning. 

Take, for instance, the wide-spread practice of ousting the oral examination with testing. That is 
not merely coercing people to keep their thinking within moulds but an obtrusion of established 
patterns, an artificial limitation of the intellectual horizon. Those under test not only cannot utter 
anything new since they are bound to reason within the suggested conceptual outlines but even do 
not begin to ponder over it, for any creative burst in such a reference frame inevitably is regarded 
as an encroachment upon the foundations of the existing knowledge, as a dethronement of 
authorities. 

Something of the kind occurs at reviewing scientific papers. It goes so far as the evaluators are 
given ready- for-use forms where the structure of review is beforehand determined, the variants of 
judgments being provided for and sometimes the weight of any component of evaluation pre- 
established. The reviewer gets driven into a certain circle of thinking beyond which it is not allowed 
to go. However self-sufficient he may manifest itself, his conclusions, all the same, will remain 
within the area of the foreseeable and admissible. Under such conditions, it seems very naive to 
reckon on the appearance of radical novelties or merely profound reexaminations of the 
established notions on journal pages or in thematic plans of research institutions. 

Of course, the “test approach” to measuring the qualification or the results obtained allows in 
most cases of discarding, at once, patently incompetent people and defective texts. However with 
the same ruthlessness it cuts off brilliant, creative personalities as well as research breakthroughs. 
In the issue, a dull bulk gets cultivated - a community of “strong specialists” doing well their day- 
to-day, routine work but not capable of flashes of inspiration and even not dreaming of them. And 
as an ideal of the growth of knowledge is taken its incremental increase with contiguity to the 
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works of the precursors but by no means with sharp shifts and, all the more, revolutions able to 
destroy suddenly the structure of knowledge formed during decades and convenient to the 
majority, and the hierarchy of scientists which serve it. 

The standardization and formalization of the procedures minimize live communication between 
people in which alone the new can be fostered. Even a genius in whose mind a truly original and 
prospective idea emerges should elaborate and polish it in personal contacts with those interested 
in that notion and capable of comprehending it. So, something radically new can grow ripe and 
get “crystallized” only in the discussions of a narrow circle of persons who have almost lost touch 
with the prevailing mentality. Their language contains something unknown to the rest of experts 
in the field but they themselves understand, albeit with effort, each other. That is, the conceptual 
world the innovators have in mind is relatively uniform, what enables a common tune to swap 
ideas (Petrosyan; 185 - 186). Otherwise any innovation would remain a germ with no ripe fruit 
growing out of. And even if - beyond expectation - the idea gets well formulated it will be doomed 
to oblivion, and scarcely someone will have it recalled until the conceptual background allowing 
of making sense of and passing it through his cognitive or practical interests emerges 

In order to get into the agenda of a fomm the new idea must pass through the fence of formal 
requirements based on an opposing conceptual material and a different line of reasoning. Clearly, 
in an ordinary case when everything goes routinely and there is no extraneous reason for making 
exception (for instance, due to protection or personal authority), that idea has no chance. It will 
have fallen in the unequal battle against the prevailing mentality and its numerous adepts. But that 
is only one side of the matter. 

The other side consists in the collegial standard that undermines the status of individuals and 
brings them to a common collective denominator. But since only few people — even among 
outstanding scientists - have minds heedful of understanding and appreciating the radically new 
ideas coming from others they inevitably turn out to be short-handed or, else, in full isolation. As 
a result, the few who could support the essential innovation often do not bring themselves to do 
so. At voting, they unavoidably lose, and many, being aware of that, even do not try to hold their 
own. On a longer distance, so kind of people cease to go into the marrow of matter and to promote 
the innovators, keeping in mind that the task is utterly embarrassing, if not infeasible. 

The role of live communication gets leveled, too. They link it usually to the lack of funding, what 
allegedly prevents from maintaining regular contacts between researchers. But to make sure that 
such an explanation is superficial and far-fetched it is enough to recall what opportunities the 
today’s means of communication afford. People - including the so-called humble ones - having, 
by their line, no need for these means uninterruptedly communicate with each other on very 
diverse occasions. The distributed (spread in space) teams are made which resolve a single problem 
in close cooperation with each other. But where the question of a broad discussion about the ideas 
being advanced arises something of the kind rarely is met with. 

Meanwhile, the atmosphere of face-to-face communication is an indispensable condition of 
emergence and growth of fruitful ideas. As early as in the very beginning of the 70s of the past 
century, in a scientific center of NASA (National Aeronautics and Space Agency), a curious inquiry 
which embraced 117 workers, including 87 “career researchers”, - 14 groups by 2 to 17 members 
in each, with the average number of 6.2 persons, - had been conducted. More than a half of the 
participants had mentioned their immediate supervisor as an important figure in the group work. 
At that, more than 60 percent saw his role primarily in encouraging right reasoning and in critical 
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evaluating the inferences and conclusions, and only less than 40 percent in advancing original ideas. 
As regards the colleagues they were considered chiefly a source of technical (substantial) 
information and “ears” to be “charged”, that is, the persons with which the “product of labor” 
should be discussed and polished. 

Moreover, rather interesting details came to light when these groups had been - in accordance 
with the results being obtained by them - divided into two categories - of high- and low- 
innovative. It turned out that unlike the non-innovative groups which were valuing their immediate 
supervisor first and foremost for his ability to advance original ideas, in innovative ones, the skill 
of critical evaluating the subordinates’ propositions was regarded as the most important merit. 
Similarly, the members of innovative groups found the main value coming from their colleagues 
to be the participation in discussing the problems and the ways of their solution (technical matters), 
while those “non-innovative” expected from them, above all, more particular and detailed 
information on the developments being conducted in the collective as a whole (Farris, 1971). 

Hence, it is quite understandable that the face-to-face communication in the course of research 
plays a key role at putting forward, discussing, and evaluating ideas. That is on one hand. On the 
other hand, the more is the degree of their newness the stronger the need for a “receptive ear” 
and concerned criticism. And to the contrary, where the working contacts between the colleagues 
are broken, crumpled up, and emasculated the chance of emergence of a new idea is far less. 
Furthermore, even when it is uttered, the ambience does not favor its refinement and polishing. 

Disregarding the Order 

The closedness and non-transparency of the procedures of evaluation in combination with their 
standardization enables the stranglehold of formal requirements and strict norms and mles 
regulating the evaluation of scientific ideas. However just such a depersonalization is frequently 
seen as a general condition of universality of knowledge, its ability to serve practical goals 
irrespective of who exacdy and in what way uses it. So, according to Merton, “universalism finds 
an immediate expression in the canon that tmth-claims, whatever their source, are to be subjected 
to pre-established impersonal criteria”. Hence entails the objectivity of scientific knowledge which 
therethrough get out of the influence of “personal and social attributes” of its creators and 
protagonists. In turn, the “objectivity precludes particularism”, forcing out the individual 
(subjective) criteria of validity (Merton, 1973; 270). It comes out that standardization and 
depersonalization of judgments about the level and quality of scientific results secure the precision 
and impartiality of evaluations. 

But even if to put aside the logical circle Merton commits in his reasoning: the universality of 
knowledge obtained owing to impersonal formalism ensures the objectivity which itself 
“universalize” that knowledge, delivering it from particularistic measures - one cannot but admit 
that seemingly scrupulously elaborated formal criteria and methods not only do not justify the 
expectations associated with them but, as a rule, are not observed at all - at least properly. Small 
wonder, the live process of weighing the merits and drawbacks and setting priorities hardly can be 
driven into the narrow framework of mechanistic standards, and its excessive formalization is to 
judgments as a straightjacket. And in order to come to a sensible conclusion the mind has to ignore 
the stiff constraints put on it. 

Such things occur not only when presenting or sizing up scientific results but also at assessing 
prospective intentions and programs, including the planning, organization, and funding of 




International Journal of Innovation, Creativity and Change, www.ijicc.net 
Volume 2, Issue 3, May 2016 



research. The formality, standardness, and non-transparency of the procedures of discussion and 
decision-making engender a picture contrary to what is required. And frequently an impression 
arises that no regulation exists there - all despite that almost every step taken by experts is 
scheduled in advance. Their activity seems to be not merely criss-crossed by a network of statutes 
and directions but also, as it is generally assumed, tightly controlled by the scientific bureaucracy. 
And any slightest departure from the behavior prescribed may be considered violation of existing 
norms and standards. Nevertheless it is nothing to say that breaches are wide-spread; they are of 
so systematic character that, watching the activity of experts assessing the research projects and 
making decisions on their support, one can conclude: they act at the command or at their own 
discretion but only not in accordance with any established order. 

Half a century ago, Roberts, an American expert who had been himself involved in the system of 
evaluation and support of research projects, subjected to a thorough examination the practice of 
financing science through the governmental funds. The conclusion he came to was shocking. 
Despite all these funds were guided, in words, by formal procedures of awarding contracts, the 
reality was so far from the “prescribed” scenario that sometimes one hardly could believe that it 
actually existed. 

The principles proclaimed by the regulations may be brought to three central ones: maximum 
competition, objective evaluation, and independent, multilevel reviewing. All they appear to be 
quite sound and favoring the effective organization of funding scientific research. However on a 
closer examination, it comes out that the actual practice poorly conforms to them. Thus, the real 
competition is much lower than that officially declared. And even when manifesting itself it is 
perfunctory and imitative. As regards the rest two principles they are observed to a still less extent. 

In 60 percent of instances, the awards were made without any formal competition. Even where it 
was provided for, the technical initiator was notifying beforehand all involved persons about his 
only desirable performer. And although nominally 7 firms in average took part in the competition, 
after the evaluation of their proposals by a small group of experts, 5 out of 6 contracts went to the 
performer approved by the technical initiator. As to that single time the “preferable” contestant 
had lost, nothing could help him. The behavior of his was thus much self-assured that he had been 
“severely reprimanded” by the technical initiator because of insulting the agency by sending in “an 
advertising brochure”’ (Roberts, 1964; 72). Something of the kind was occurring also in more 
complicate cases. The initiators’ preferences were eventually finding support, though not so 
obviously. 

The evaluation objectivity must be secured through “numerical” measuring the projects, by a team 
of experts. But it sounds like a tall story. As Roberts had fairly observed, “only the naive 
nontechnical man can believe that technical performance and — more difficult — technical 
proposals can be “objectively” evaluated.” In research and development, “only subjective 
evaluation is possible.” Not only hypotheses and explanations are called into question but even 
technical “facts” become “subject to dispute by competent evaluators.” As to “technical opinions 
on yet unproved research and development projects” they “can be debated still more. We can 
expect”, Roberts concludes, “honest appraisals by competent men, but this must reflect their 
experiences, judgments, technical prejudices, and other factors of a subjective nature” (Roberts, 
1964; 74). Any attempt to objectify the evaluation will bring only to imitation of objectivity, to 
adjusting it to external requirements, not to changing its nature. 
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However the inner scale the expert puts on the subject may get formalized and standardized, the 
main thing in the procedure of evaluation remains how he sees and understands its substance, or 
- more precisely - how it gets passed through the alembic of his conceptual world. But that inner, 
entirely personal world is insuperably subjective. Not in vain, 9 out of 10 expert groups evaluating 
the applications for non-military contracts admitted that they were setting their numerical estimates 
after general discussions on the merits and demerits of the contenders. So, the quantitative 
characteristics actually did not measure the applications but, rather, confirmed, in retrospect, the 
agreement obtained. They served not so much a ground for decision-making as a subsequent 
justification of the already taken stand. 

The multilevelness of reviewing is out of whack at all. In the case of contracts to the amount of 
$1 million, formal procedures had not been employed; everything was done in a simplified order. 
But even when the sums had exceeded that threshold the reins were remaining in the hands of the 
technical initiator, while the groups on the higher levels actually were rubber-stamping his 
recommendations. So, only in 1 out of 10 contests on non-military research and development, folk 
in Washington decided otherwise than those who had passed the initial verdict and even that 
exception took place due to a serious split among the experts as to the job descriptions, and any 
agreement on the lower levels seemed to be infeasible. 

Under such conditions, there is no talking on the independence of evaluators. Indeed, is there a 
reason to demand it from dependent persons who, on a quirk of fate, found themselves dressed 
in the judge gowns? As one of them admitted behind the scene, “they tell me that when I’m on an 
evaluation team, I’m supposed to be Joe Independent. What they forget is that for the rest of the 
year I’m Joe Subordinate, and if my boss says ‘Change it’, he’s the boss” (Roberts, 1964; 75). Such 
a position is the manifestation of the self-preservation instinct. For, the fulfillment of the 
requirement of independence “in patches” leads to a functional schizophrenia. The independent 
subordinate is about the same as a cook governing a state. 

Thus, of three principles declared - competition, objectivity, and independence - only competition 
possesses a practical meaning. Independence is completely impossible under the hierarchy holding 
sway over the scientific community. At the best, it degenerates into a self-deception, but at the 
worst serves an instmment for securing the self-seeking interests. As to objectivity it only 
consolidates the leverages in the hands of those at the summit of the pyramid and deprives those 
being piedmont, of the opportunity to somehow influence the “conventional” judgments and 
evaluations. 

“Communal” science? 

There is nothing worse than a farfetched objectivity (“detachment”). For, actually under guise of 
it, instead of one (somebody else’) subjectivity another (own) is railroaded. And instead of 
admitting that two subjectivities come into collision, and the winner must emerge from honest 
competition - openly, publicly, and according to clear and comprehensible mles when the both 
parties get an opportunity to explicidy present their positions and reasons for them, - one party 
proclaims the opposing (alternative) subjectivity not enough objective to reach the standards of 
the description of reality, that is, the level its own (established) subjectivity allegedly stays on. In 
this regard, the principle of objectivity serves nothing but a tmncheon for putting down the human 
imagination and creative impulses. 
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But if to add to the objectivity also the principle of “communality” which becomes increasingly 
popular among those working in science and studying it the human thought will have been entirely 
caught in toils. Scientific thought will have fallen into full dependence on the prevailing tendency, 
and any dissent - up to the most trifling - will be impossible and unthinkable. To make sure of 
that, it is enough to get familiarized with the wide-spread treatment of science as a social organism 
whose activity depends not so much on the conceptual heritage learnt by scientists or on what they 
study as on the organization of work and their mutual communication. Brought to its logical 
completeness, this approach turns into a funny caricature having few junctures with reality. 

Thus, in a book with a characteristic title “Real Science: What It Is, and What It Means” science 
appears to be a kind of collective economy with common ownership of the products of labor and 
joint evaluation of their merits. Sure, each researcher has the right to utter his own judgment, but 
it cannot have any real weight unless gets general support. The research results, before they are 
accepted for publication, must be approved by experts. “The norm of communalism”, Ziman, the 
author of the book, states, implies that “for an item of information to be acceptable as a potential 
contribution to science, it has to reach a minimum standard of credibility and relevance.” 
Moreover, “it has to be presented in a form capable of undergoing further communal tests before 
it counts as ‘scientific knowledge’ in the fullest sense” (Ziman, 2003; 85). Without that, the yield 
gathered - scientific findings - cannot be not merely sorted out (with determining a place for them 
within the body of knowledge) but even stored in the granary (included in the composition of 
science). What precisely has grown is defined not by the tree the fruit is picked off and not by how 
it looks like but by a free decision of the community. And should its members - beyond 
expectation - “ascertain” that to be a weed they will all at once throw themselves to root it out at 
a breakneck speed. 

Some go still further, making the scientificity and reliability of the results obtained conditional not 
merely on collective decision of the researchers’ community but on the mood of its elite. So, in 
Mack’s words, “a new idea becomes scientific and engineering knowledge only by being accepted 
by enough influential scientists and engineers” (Mack, 1990; 2). It comes out that the status of an 
idea in science is barely marginally determined by its own virtues. And even the popularity among 
wide circles of researchers does not communicate to it some sufficient weight. Only due to the 
support of the “pillars” of the community, a piece of knowledge gets included in the treasury of 
science. At that, the place thus occupied is identified with explanatory power and heuristic 
potential. 

But even more funny looks another thing. This “communal” notion of science as the realm of 
arbitrariness of collective subject is combined in a strange manner with the demand of eliminating 
all the subjective. Although the observation is individual by nature it must give up that individuality 
to become tmly communal. The substance of the scientific method consists just in “the eradication 
of subjective influences on research findings” (Ziman, 2003; 87). But how the objectivity of 
scientific knowledge should be combined with the impassable subjectivity of labeling a piece of 
information as belonging to science? And why the knowledge already at birth rid of any subjective 
taint should be afterwards subjectively assessed for objectivity? Clearly, that giddy logic is called 
upon not to substitute the subjectivity for objectivity but, rather, to replace the individual 
subjectivity with a collective one. 

Objectivity treated in such a way is, of course, poorly compatible with the independence of 
evaluations. The main thing is here the commitment to the established, beforehand known criteria. 
Consequently, this kind of objectivity represents a dependence “by definition”, connectedness 
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with the existing, facedness to the past. But the objectivity subjecting the knowledge to the 
“communal” mind inevitably diverts from the future, fetters the new, and restrains the flight of 
thought. 

Meanwhile, real objectivity is not impartiality or disinterestedness but just freedom from 
“communal” chains. Not to mention both disinterestedness and impartiality to be actually 
unachievable, it must be admitted that they not merely do not favor the search of truth but, rather, 
bother it. Impartial and disinterested mind is not able at all to come to any result, for cognition 
itself starts from interest (passion). Only the mind absorbed in the subject under examination, and 
considering it through the lens of personal affections and practical needs is capable of building the 
picture (knowledge) which afterwards can turn out to be true, that is, let fulfill the goals set. On 
one hand, objectivity is not a cognitive attitude but a feature of knowledge (engagedness in the 
object) manifesting itself when applying to practical tasks; on the other hand, it implies some high 
degree of interestedness in and partiality to the object - ideally up to full fusion. 

Meanwhile, in its “communal” meaning, objectivity is impartiality of the subject to the object but 
partiality to the place he occupies within community; independence from the conditions of 
knowing but total dependence on the ambience he has to work in. Something objective is here not 
that engaged in the object but that what is subjected to “communal” norms having imbibed the 
past experience of other subjects making the community, that is, ultimately to others’ subjectivity. 
Naturally, under these circumstances, the original idea turned to the object, directed at making 
sense of it despite the “crystallized” subjectivity and even opposing to the latter should not reckon 
on the support from people around. Quite the contrary, it will be pressed out of “communal” 
mind in every possible way as a force undermining the grounds of the established knowledge. 

Everybody who has, once at least, taken part in contests for grants or scholarships, or defended a 
dissertation knows very well that the main rule one should obey is not to make tongues wag and 
not to claim to significant originality. Otherwise he will not only be misunderstood and arouse 
some unhealthy interest to himself but also call forth harsh resistance. This applies to both young 
and quite “moulded” researchers who have yet not had time to get “bronzed” and take a 
foreground place in the science hierarchy. For, exposing their insights, they, intentionally or not, 
focus the attention of the potential critics just on those points which appear to be the most 
vulnerable to the dominant mentality. 
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Fig. 3. Communal objectivity as compared to real. 

“Communal” objectivity - impartial, disinterested contemplation of the object by 
individual mind under the guidance of collective mentality, or — more intrinsically - of 
rather few “hierarchs” who set the tone (the canon) of what is generally accepted. 

Real objectivity - interested and partial engagedness by individual mind in the object 
(ideally joining with it) passed through the alembic of collective mentality as engrained in 
human experience and established knowledge. 

Not in vain, as back as in the beginning of 80s of the past century, Armstrong has stated his 
effective “author’s formula” for those wanting to have quickly and swimmingly published their 
papers. It comprised 6 obligatory points: not to touch on important problems; not to encroach on 
the prevailing notions; not to produce unexpected results; not to employ simple methods; not to 
disclose all the details; and not to write clearly (Armstrong, 1982). Fulfilling these requirements 
lets lower the keenness of perception of the text by reviewers and reconcile them with the thought 
that it can be read by other people. In the issue, the material comes out assuredly. 

In spite of all the Armstrong’s irony, his proposition has a sizable practical meaning. And keeping 
to rules formulated by him, one really can make easier his getting to journal pages. However the 
trouble is that such a publication is devoid of any scientific purport - sure, if not to consider so 
solving the carrier problems or flattering one’s own vanity. For it does not contain any accretion 
of knowledge. 
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Thus, all-out meeting what the experts actually demand from authors, and what, in turn, the so- 
called standards demand from the experts would bring only to “smoothing” the scientific mind, 
nurturing mediocrities, cutting off the inventive and the talented, and above all, to the virtual 
exception of serious innovations. At the best, only minute improvements would be encouraged, 
there would be some room barely for the “further development” of what is well-known, trite and 
hackneyed, for the additional confirmations of the “unshakeable” principles. In researchers, from 
the cradle would be suppressed the originality, and they would be put out of the generation of 
ideas. 

The Citation Indexes 

When measuring the contribution of a scientist to knowledge, many start from his list of published 
papers. Particularly desirable appears to be their coming out in so-called top-tier journals or, at 
least, in those being considered respected in the institution the author has dealings with. It is so 
long since. And though the belief in such a simple “yardstick” is already essentially undermined, it 
is still widely used at assessing the merits of a researcher and at assigning him to a position. 

Meanwhile, the list of published works as a ground for substantial conclusions is too shaky and 
unreliable. Indeed, even different papers of the same researcher in the same edition greatly vary in 
richness of content and in scientific level. Then, what to say on the works of different authors, 
which appear in very diverse journals? Is it reasonable to equal a “policy article” or a report on 
unexpected results of a major experiment with a “stock” abstract or a note in a popular journal? 
But many researchers do not feel shy to indicate in their accounts just so publications. That is why 
these lists contain, beyond doubt, incommensurable “points” as to their value - from openly 
slapdash workmanship to true masterpieces - which by no means may be put on the same level. 

Lately, instead of the list of published works, the quantity of their citations by peers little by little 
comes to the forefront. It is presented as a measure of the value of papers and their authors. The 
more they are mentioned the higher their importance to the development of knowledge. 

Such a “maximalistic” interpretation of the citation indexes has a long-standing tradition. For 
instance, one of its protagonists Garfield believed that with aid of this tool, it becomes possible to 
identify the value of literature to the scientific community. Is it worth tolerating the costs and 
difficulties connected with search and retrieval of sources, trying to miss nothing of what is already 
made and not to repeat earlier researches? - he asked, having in view that the index suggested gives 
an opportunity to separate the wheat from the chaff (Garfield, 1964; 649). His colleague Maddox 
set the question even more pointedly: “Is the literature worth keeping?” (Maddox, 1963; 14), - 
evidently, thinking that the texts which receive no or little references are a deadweight on the 
shoulders of mankind and are to be removed from the treasury of knowledge. 

Meanwhile, the citation index initially had been considered barely as one of the methods of 
searching the literature on a given topic. In Garfield’s words, “a citation index is an ordered list of 
cited articles each of which is accompanied by a list of citing articles.” With aid of it, one can 
identify “what subsequent papers have cited a particular reference” (Garfield, 1964; 651). 
Therethrough, a transition is secured from some papers to others going into the same issue and 
touching upon the allied problems. Elsewise, due to the citation index, a thematic cluster comes 
into being, the acquaintance with which allows of getting a rather explicit and systematic notion 
of the subject. And therefore, to the researchers having to thread their way through information 
noises and to bridge conceptual gaps, it turns out to be a very useful methodic support. But that 
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by no means attaches to the citation index any methodological role directing the research proper 
and, all the more, does not endow it with an evaluative potential letting rate the findings and the 
publications comprising them. 

Garfield himself, too, was aware of the fundamental flaws, “birthmarks” of his citation index. As 
he has admitted, the utility of it entirely depends on how precise is the bibliography presented in 
the papers. When references are incomplete and do not embrace a part of relevant publications, 
or distort the thematic cluster, referring to the papers which are little conductive to making sense 
of the problem, a danger emerges to disparage some works and overestimate others. 

However, Garfield saw there no insuperable difficulty. It seemed to him that there existed a simple 
and efficient way to obtain relevant references in full. For that, “in evaluating papers submitted to 
journals, referees should determine whether all pertinent references have been provided. The 
citation index will significantly assist the referee in identifying such pertinent references” (Garfield, 
1964; 652). But whence it appears which of references are relevant as well as which of them one 
may do without? Someone traces his research to these sources, while another connects the results 
obtained with different works. Demanding from the author to make changes and additions in the 
bibliography makes him deform the course of his own thought. But it means that the history of 
the problem field gets recounted arbitrarily and driven into a beforehand prepared mould, not to 
mention that under such a guise, it is rather easy to strike out from the text disliked names and 
works, and on the contrary, shove in it “decent” ones. 

How to slip out of this trap? 

The answer offered by Garfield is marvelously simple: the reviewers must rely on the same citation 
index — just it serves as a criterion allowing of distinguishing the pertinent from the impertinent. 
At that, Garfield does not notice the vicious circle in his reasoning. In order that the citation index 
may precisely reflect the significance and usefulness of the publication it must take in all relevant 
citations. But deciding on which of them are relevant is admissible only on the base of the same 
index. Hence, any new work gets passed through the alembic of the old index and cast away when 
does not fit into it. Therefore, the old prejudices become the measure of value for new insights. 

Those who realize that original ideas get learnt not at once but with a certain delay, try to save at 
least the weak version of the citation status. They are aware that the citation index does not allow 
of catching the value of the works containing new knowledge. Yet, it enables “to expose those 
papers of major scientists, which are, for some reason, little cited” and to ascertain wherefore that 
occurs. Tme, “it remains unclear how one can expose interesting papers of non-prominent 
scientists.” But this circumstance embarrasses little, for the “non-prominent” should draw no 
heightened attention. Not in vain, as virtually the only factor determining the non-proper 
understanding of such works are named “the defects of presentation” (Nalimov, Moulchenko, 
1969; 117 - 118). That is, those papers are little cited not because the “prominent” colleagues 
cannot go deep into their marrow but on the score of flaws of the text itself. 

However the snag is that no scientist is “prominent” from birth. Those with such a status have 
obtained it through getting overgrown with references which, in turn, emerge as they publish their 
results. But how to reach the top, generating mainly breakthrough ideas which are not enough 
accessible to people around and hardly learnt even by the nearest colleagues? It comes out that the 
beginning talented researcher full of original intents has only one way to go: at first, to publish 
“standard” papers - variations on hackneyed themes - and only after that, on finding himself 
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within the cohort of “prominent” fellows, to set about promulgating the new conceptions carrying 
a considerable accretion to scientific knowledge. But it does mean that the researcher gets added 
on to the elite, having made virtually no contribution to science and only thereupon as if acquires 
a right to utter his own word. Elsewise, it is not new ideas that make him “prominent” but, to the 
contrary, just after gaining the status, he gets an opportunity to expound them and be heard, at 
that. 

Does it mean that there is no relation between the number of references to the paper and its 
substantive evaluation? Surely no. Empirical research reveals a positive, statistically significant, 
though not perfect, correlation of the experts’ judgments on the publication and the average 
number of references to it. Nevertheless it is clear that at such a distribution of data, the quality of 
paper can be neither the single nor even the chief reason for high citability. It is characteristic, too, 
that the strongest correlation is registered in the field of “curiosity-driven” basic research where 
exists a vast room for conceptual parallels and associations. As regards the applied areas there 
much less dependence of references on the status of papers is observed (Rinia et al., 1998;105). 
No wonder, the live reality make experts more frequently look out of the window and be not too 
much bewitched with the magic of names and the force of tradition. 

Quite noteworthy is also the revealed correlation between self-citation and the colleagues’ 
judgments. The authors consider it to be related to one of the two possible causes - the self- 
isolation of the researcher or the unique nature of his work. In the first case he works beyond the 
framework of the existing scientific groups, while in the second the search is conducted far away 
from the interests of others and therefore gets not favored with their attention. Anyway, such a 
researcher has extremely few chances of high evaluation on the part of colleagues (Rinia et al., 
1998;105). But unfortunately, another and perhaps even more important possibility slips the 
authors’ mind. 

The researcher may stay within the “scientific organism” and work out a “burning” problem, but 
when he goes so far ahead that his conclusions turn out to be on or, all the more, behind the edge 
of the comprehension of contemporaries his bibliographic indexes will scarcely reach some 
conspicuous level. For, from the stand of common sense, it is the same self-isolation as the 
reclusion - be it organizational (refusal to collaborate with the colleagues) or conceptual (retreat 
into the fields of no broad interest). 

So, what speaks well the citation index for? 

In itself — for nothing. There are a lot of reasons making the authors mention colleagues. As Gilbert 
observes, by the peculiarity of citing, one can, rather, single out a community of researchers well 
known to each other, studying the adjacent problems and employing close patterns and methods 
of reasoning. Therefore, it would be more correct to consider the style of referencing a 
manifestation of the adopted tactics of persuasion (Gilbert, 1977; 113). Gilbert notices that authors 
resort to references quite selectively, aligning, first of all, with how the latter could corroborate 
with the position being uttered. Hence, they prefer “important and right” sources. The “wrong” 
materials are mentioned, too, but only for running the gauntlet of criticism. Finally, the “trivial” 
and “non-pertinent” works are avoided not to overload the text and not to attach to it a taint of 
lightness (Gilbert, 1977; 1 16). Sure, the author can be in the fault and not produce a due impression 
with the names and works mentioned. But one way or another, the sources which, in the author’s 
opinion, scarcely will be useful in promoting his own paper are cited only in very rare instances. 
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Furthermore, granted that the citation in itself could talk of the significance of publication it would 
be, all the same, extremely hard to make of that a measure for the merit of research. 

First, many references are outright “made-to-order”. Some journals nearly wrench out the authors’ 
(particularly young ones’) hands to force them to mention in the papers submitted the members 
of the board, reviewers or other persons of their circle. The candidate for a degree almost always 
is bound to cite his research supervisor and official opponents as well as the members of the 
scientific council conferring the degree. And in general, the authors much more readily mention 
those of the “same kin”, having in view they will respond in kind. 

Second, even if to suppose that the references in the text are candid and quite pertinent their 
significance is far from being identical. Amongst them occur both those of great importance and 
petty ones, sometimes dispensable at all, touching on the subject being studied only marginally. 
Within a research, 706 references from 30 theoretical articles on high-energy physics issued by the 
journal “Physical review” during 1968 - 1972 had been analyzed. More than 40 percent of them 
turned out to be quite “impertinent”. Can they attach to the paper the same weight as that coming 
from relevant ones? 

And third, a good many of mentions bear negative or critical character. In the same research, 14 
percent of so instances had been detected (Moravcsik, Murugesan, 1975). Along with the indirect 
recognition of the contribution to the working out of the problem, this kind of citation exposes 
the flaws and lack of professionalism the publication suffers from. Nevertheless, formally such a 
reference, too, should be clocked up for it. Understandably, in the “negative” instances, it is out 
of question to acknowledge a real accretion of knowledge, originality, or practical potential coming 
from it. However they not worse than other references “inflate” the researcher’s rating which is 
assumed to be the basis for evaluating his creative power. 

Why then, despite the evident flaws of this practice, it remains still in force and is used in assessing 
the scientific brainpower and the results obtained? 

All is quite elementary. The employer or another party interested in the research of a scientist — 
particularly as to theoretical and general problems - are concerned not so much about the 
achievements in themselves it is reasonable to reckon on with him as about the echo they can 
receive due to his publications. And in this regard, the mediocre papers which will securely appear 
in respected journals, obtain prestigious awards, or draw the attention of the general public are 
much more valuable than innovative breakthroughs risking to meet with no proper understanding 
in the scientific community and, thereby, to bring to the institution the author works for only 
needless headache. 

That will be so while in the focus is not the substance but its surface attributes, not the ideas 
presented in papers which undergo evaluation but where they are published, whom cited by, and 
what the rank of those supporting them is. Meanwhile, to ascertain the tme potential of a 
researcher or the quality of his papers it needs to address immediately to their substance and to 
discuss, without excessive formal restrictions, the results they comprise. All the rest evaluations 
are secondary and can serve only as complements, not a firm ground for making decisions. 

The main task of any researcher consists in producing new knowledge useful in the broadest sense 
of the word. The point should be not only and perhaps not so much of purely utilitarian 
applications responding to the vital needs of society as of opening new horizons, deepening the 
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understanding of the world around, and working out the ways of its adjustment to human wants. 
But just the execution of this function is hampered by the established practice of appraisal of 
scientific findings. 

Concluding Remarks: The Disease Diagnosed 

What lessons can be drawn from the above analysis? 

Putting aside the nuances and details, the defects and flaws of the current system of evaluation of 
scientific results may be outlined in such a way. 

1 . The “blind” reviewing by no means secures the unprejudicedness of judgments 
and the adequacy of assessments. It barely protects the reviewer from blames, as if placing him in 
a semitransparent (with a one-way visibility) box where he can afford any excess without being 
concerned about both the authors’ and the general public’s response. Meanwhile, that is the best 
breed ground for arbitrary decisions because delivers the evaluator from any responsibility for the 
verdicts being passed. 

But the reviewer scarcely would get the power over researchers should those behind him be not 
interested in it. Tampering with reviewer, the institutions he represents (journals, prize committees, 
commissions making appointments) get the desired without exposing themselves to light. Having 
no direct relation to the author or candidate and formally basing on the reviewer’s judgments, they 
obtain, ultimately, the outcome planned. The reviewer readily joins that play, as well, even when 
he is quite aware of manipulations, for, acquiring a certain status and an opportunity for self- 
expression and flattering his own vanity, he runs, at that, no risk because remains invisible and, 
therethrough, invulnerable. 

2. When assessing the quality of scientific work, reviewers frequently emphasize the 
source material (data, experiments, the literature used), the instruments employed (methods, 
reasonings, calculations), or the inferences obtained (final statements). However it does not reveal, 
in whole, the quality of research as focuses on particulars and misses the main point. To boot, as 
practice shows, experts regularly overlook even overt errors and defects, not to mention barely 
noticeable lapses, though easily detect flaws which actually do not exist there at all. What is, then, 
the subject of evaluation? 

It is assumed that the experts are called upon to determine the scientific level of the work under 
review, by which commonly is meant the conformity with the established norms, substantiatedness 
(the linkage to the “canonical”, justified knowledge), and the erudition of the author himself. 
However the evaluation made on the base of such criteria favors the progress of science not in the 
least. For, the purport of research consists in the creation of new knowledge which unavoidably 
opposes to the “proven” approaches to evaluation. And the deeper and larger the newness the 
farther it goes beyond the comprehension of experts. Consequently, only the quality of that 
research is amenable to a “reasonable” expertise which contains only a tiny little increment of 
knowledge, that is, has almost no scientific value. 

3. The standardization of judgments and assessments gives the priority to those 
findings which, to a great extent, comply with the established standards. And that entirely proves 
itself when the point is of routine material or of ideas not saturated with newness. But as soon as 
the measure of non-ordinariness exceeds the “tolerable” level, that is, oversteps the edge of “fore- 
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seeable”, such a knowledge is taken in as something inadmissible and, therefore, “unconditioned” 
or off-grade. To the standards, it is of no matter from what side the breach comes, and is the 
person disregarding them a genius or an ignoramus. What does not fit into the standards is rejected, 
and in this respect, deep insights seem to be subquality to the same degree as crude reasonings. 

The “common denominator” (something averaged) is very appreciable in craft, because sets its 
canon. But where the purport of activity is to discover the unknown and, therethrough, to 
overcome the established, an unsolvable controversy arises between the aim and the image of 
result. The new, that is, the unfamiliar and unexplored, should be moulded by dint of established, 
that is, wonted and tested, forms. But it is feasible - even at a stretch - only in the case of 
insignificant novelties which, though coming out from under the gauge, do not demolish it and 
dilute its scale (reference frame). In the case of breakthrough ideas, there exist no established forms 
they can be represented in, and, consequently, the expert has nothing to confront them with. That 
is why the importance of live communication which alone is capable of grasping the most subtle 
shades of thought and surfaces what is not yet entirely realized by the author himself increasingly 
gets up. 


4. The formalization of evaluation, that is, making the impersonal norms its 
cornerstones, is fraught with self-erosion, what eventually brings to that they cease to be observed 
at all, and the “leveling” of evaluator actually imparts to him almightyness. He takes complete 
control of the process, bearing no responsibility for it because acting in the name of a “soulless” 
mechanism. Of three main principles by which the management of science is supposed to be 
guided - competition, objectivity, and independence - none works in full measure. And therefore, 
instead of an impersonal formalism which allegedly must be a manifestation of justice, order, and 
effectiveness, personal discretion holds sway over all - often unjust, disorderly, and inefficient. 

The systems of evaluation of scientific research are fundamentally hierarchical, and, consequently, 
they do and can contain no genuine independence. The demand of objectivity placed on 
primordially subjective judgments turns out to be a self-deception and disguises, in fact, the 
domination of parties having influence and opportunity to intervene behind-the-scenes with 
making decisions. As to competition, in view of lack of independence and the raising of the 
subjectivity of particular persons to the level of universal objectivity, it may be only of imitative 
nature. Even in the cases of rivalry, all is orchestrated by those disposing power and retaining the 
last word. 

5. If to add to this peculiarly understood objectivity the principle of “communality” 
consisting in not merely group organization of scientific work and its subjecting to the common 
task but also in adjustment of individual approaches and construals to collective patterns and 
stencils (standardized views of the world) the dependence on the dominant tendency becomes 
absolute. Flashes of inspiration and breakthrough ideas vanish from the communal mentality, and 
going out of its bounds turns to be virtually prohibited. Live thought gets tightened from two sides 
at once. On one hand, it is transilluminated through the alembic of the canonical knowledge and 
fit into it; on the other hand, the canon itself is set not by all or at least the majority of researchers 
but by few “pillars” who have reached their positions not necessarily due to contribution to science 
and, by the time of their “canonization”, have almost left active research practice. So, setting the 
horizon of knowing, the “high priests” cut off from the live fabric of creativity disallow the 
“humble” researchers to see what is beyond that artificially drawn line. 
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Thus, the objectivity of “communal” science is not engagedness in the object (at least partial 
coalescence with it) but submission to the collective mentality or - to put it more precisely — to 
the collegial subjectivity of its “hierarchs”. That is why it seeks to have an impartial and 
disinterested attitude to the object. Meanwhile, such impartiality actually turns into a partiality to 
the place the researcher occupies in the professional community and in society as a whole, just as 
the independence from the object - into total dependence on the ambience he works in. 
Therethrough, his own imaginary objectivity proves to be, ultimately, somebody else’s subjectivity 
- at that, not live, pulsating but objectified and stiff and, in this sense, dead. It is tied not so much 
to future as to past, being a stranglehold on fantasy and creativity. 

6. Both the list of publications and citation indexes of a researcher tell, in themselves, 
nothing as to his contribution to science. The linkage of it to this factors not only does not improve 
the quality of research but, to the contrary, lowers it, for, on one hand, exceedingly emphasizes the 
quantitative side of the matter - in prejudice to its level of quality and, on the other hand, implies 
the highest possible proximity and comprehensibility of the results achieved, to people around - 
and first and foremost, to peers, what primordially limits the degree of newness the ideas being 
advanced contain. To publish as much as possible works which are, to boot, largely referred to 
one must to have them clear, if not to say — pellucid, easily accessible and not controversial. But it 
means they should have little essentially new. 

The circumstance that these factors, despite all their barrenness, are used as probably the main 
criteria of evaluation of researchers and their contribution to science is accounted for simply. They 
are purely formal and easily measurable, while the relevant but difficult tasks are frequently 
substituted for another - easily performable but having no direct relation to the point (it is human 
to look for not where the thing has been lost but where the light is brighter). This is one side of 
the coin. The other - and more important - consists in that those doing evaluation are interested, 
in fact, not so much in the results as such as in the echo they produce both in the scientific 
community and among the general public. That is why they have nothing to do with the great 
breakthroughs which yield no dividends in the foreseeable future. And on the contrary, even the 
ideas known to be purposeless will get glorified if they are expected to fetch for interested parties 
some practical effect (authority, funds, et al.). Such an attitude does not encourage creative efforts; 
it, rather, undermines them. But if there is a striving to support conceptual breakthroughs nothing 
is left but to refuse simple, bureaucratically handy criteria and address the substance of research 
results - without putting profound ideas on a par with perfunctory, comprehensive with shallow, 
and prospective with trivial. 

The evaluation of scientific findings is not a strict procedure but, rather, an art implying not so 
much comparison of them with the established guides and criteria or the demonstration of their 
consistence with or divergence from the existing knowledge as the revelation of conceptual and 
practical vistas they open. To cope with the task properly it is necessary not merely to know a lot 
and to be experienced and well-versed in high-level research. Much more important is to have 
scientific flair and taste, broad vision and far-reaching intellectual horizon - in order to be able to 
draw a thread from the subject under evaluation to the complex, sophisticated, and 
multidimensional context of science and — all the more — to the promising lines of its future 
progress. It needs more intuition and insight than formal and plain methods. 

The context of evaluation in modern science seems to be an impasse having no overt and manifest 
ways out. Where to look for them, and what changes to make to overcome the collisions and 
discrepancies? These questions are to be answered in the second part of the paper. 




International Journal of Innovation, Creativity and Change, www.ijicc.net 
Volume 2, Issue 3, May 2016 



References 

Armstrong J. S. (1982). Barriers to Scientific Contributions: The Author’s Formula // Behavioral 
and Brain Sciences. Vol. 5. N 2. P. 197 - 199. 

Barber R. J. (1966). The Politics of Research. Washington (D. C.): Public Affairs Press. 

Farris G. F. (1972). The Effect of Individual Roles on Performance in Innovative Groups / / R&D 
Management. Vol. 3. N 1. P. 23 - 28. 

Garfield E. (1964). “Science Citation Index” - A New Dimension in Indexing / / Science. Vol. 
144. N 3619. P. 649 - 654. 

Gernert D. (2008). How to Reject Any Scientific Manuscript / / Journal of Scientific Exploration. 
Vol. 22. No. 2. P. 233 - 243. 

Gilbert G. N. (1977). Referencing as Persuasion / / Social Studies of Science. Vol. 7. N 1. P. 113 

- 122 . 

Griggs J. (2008). The Missing Piece // Edit: The University of Edinburgh Alumni Magazine. 
Summer. P. 16 - 17. 

Hartwell L. H., Culotti J., Pringle J. R., Reid B. J. (1974). Genetic Control of the Cell Division 
Cycle in Yeast // Science. Vol. 183. N 4120. P. 46 - 51. 

Horrobin D. F. (1990). The Philosophical Basis of Peer Review and the Suppression of Innovation 
/ / Journal of the American Medical Association. Vol. 263. N 10. P. 1438 - 1441. 

Mack P. E. Viewing the Earth: The Social Construction of the Landsat Satellite System. Cambridge 
(MA): MIT Press, 1990. 

Maddox J. (1963). Is the Literature Worth Keeping? / / Bulletin of Atomic Scientists. Vol. 19. N. 
9. P. 14. 

Merton R. K. The Sociology of Science: Theoretical and empirical investigations. Chicago; L.: The 
University of Chicago Press, 1973. 

Moravcsik M. J., Mumgesan P. (1975). Some Results on the Function and Quality of Citations / / 
Social Studies of Science. Vol. 5. N 1. P. 86 - 92. 

Nalimov V. V., Moulchenko Z. M. (1969). Scientometrics: The study of development of science 
as an informational process. M.: Science. 

Osburn C. B. (1989). The Structure of the Scholarly Communication System / / College and 
Research Libraries. Vol. 50. N 3. P. 277 - 286. 

Peters D. P., Ceci S. J. (1982). Peer Review Practices of Psychological Journals: The Fate of 
Published Articles, Submitted Again / / Behavioral and Brain Sciences. Vol. 5. N 2. P. 187 - 195. 




International Journal of Innovation, Creativity and Change, www.ijicc.net 
Volume 2, Issue 3, May 2016 



Petrosyan A. E. (2015). Within a Nutshell: The Mental Roots of Human Insusceptibility to New 
Ideas // Journal of the Knowledge Economy. Vol. 6. N 1. P. 157 - 189. 

Pringle J. R. (2013). An Enduring Enthusiasm for Academic Science, but with Concerns // 
Molecular Biology of the Cell. Vol. 24. N. 21, P. 3281 - 3284. 

Rinia E. J., Leeuven T. N. Van, et al. (1998). Comparative Analysis of a Set of Bibliometric 
Indicators and Central Peer Review Criteria: Evaluation of Condensed Matter Physics in the 
Netherlands / / Research Policy. Vol. 27. N 1. P. 95 - 107. 

Robbins B. (1996). Social Text and Reality //In These Times. July 8. P. 28 - 29. 

Roberts E. B. (1964). How the U.S. Buys Research / / International Science and Technology. Vol. 
4. N 33. P. 70 - 77. 


Sokal A. D. (1996). Transgressing the Boundaries: Toward a Transformative Hermeneutics of 
Quantum Gravity / / SocialText. Vol. 14. N 1 — 2. P. 217 - 252. 

Sokal A. (2010). Beyond the Hoax: Science, Philosophy, and Culture. Oxford: Oxford University 
Press. 

Schroter S., Black N. et al. (2008). What Errors Do Peer Reviewers Detect, and Does Training 
Improve Their Ability to Detect Them? / / Journal of the Royal Society of Medicine. Vol. 101. N 
10. P. 507 -514. 

Taschner R. (2007). Erosion von Wissenschaft / / Erwaegen - Wissen - Ethik. Bd. 18. H. 1. S. 58 
-59. 


Thurner S., Hanel R. (201 1). Peer-review in a World with Rational Scientists: Toward Selection of 
the Average / / The European Physical Journal B - Condensed Matter and Complex Systems. Vol. 
84. N 4. P.707-711. 

Ziman J. (2003). Real Science: What It Is, and What It Means. Cambridge (MA): Cambridge 
University Press. 

Zuckerman H. (1977). Scientific Elite: Nobel Laureates in the United States. New York: Free Press. 




